Using a digital image in a social networking system

ABSTRACT

Systems and methods for using a digital image in a social networking system may use digital image to identify a physical entity. Information about the identified physical entity may be provided to an electronic display for review by a user. The user may perform a social networking action with the identified physical entity or a website associated with the physical entity. Social networking actions may include rating or commenting about the physical entity or the associated website via a social networking system. Social networking actions may also include sharing information about the physical entity or associated website with another user via the social networking system.

BACKGROUND

The present disclosure relates generally to using a digital image in asocial networking system. The present disclosure more specificallyrelates to receiving a social networking command regarding a physicalentity in the digital image and/or a website associated with thephysical entity.

Generally, online social networking systems allow users to share theiropinions about any number of different topics with other users. Forexample, a user may rate up an online article that they enjoyed reading.Other users may be drawn to the article based on the number of peoplethat also positively rated the article. A social networking system canalso employ a suggestion mechanism. In such a case, a positive ratingfor an article by a user may result in the article being suggested toothers in the user's social networking groups.

SUMMARY

Implementations of the systems and methods for using a digital image ina social networking system are described herein. One implementation is acomputerized method for using a digital image in a social networkingsystem. The method includes receiving, at a processing circuit, arequest to log into a user profile of the social networking system. Themethod also includes receiving, at the processing circuit, a digitalimage of a physical entity. The method further includes associating thedigital image with the user profile. The method yet further includesperforming, by the processing circuit, image recognition on the digitalimage to identify the physical entity. The method also includesretrieving, from a memory, the name of the physical entity and anindication of a website associated with the physical entity. The methodfurther includes providing entity data over a network, the entity databeing configured to cause a graphical user interface to be displayed byan electronic display. The graphical user interface includes the name ofthe physical entity, the indication of a website associated with thephysical entity, and one or more inputs configured to receive a ratingfor the website associated with the physical entity. The method alsoincludes receiving, at the processing circuit, the rating for thewebsite associated with the physical entity, the rating being associatedwith the user profile. The method further includes providing anindication of the rating for the website to an electronic device basedon whether the electronic device is logged into a user account sociallyconnected to the user account associated with the rating.

Another implementation is a computerized method for using a digitalimage in a social networking system. The method includes receiving, at aprocessing circuit, a digital image of a physical entity. The methodalso includes identifying, by the processing circuit, the physicalentity in the digital image. The method further includes providingentity data for the identified physical entity. The entity data isconfigured to cause a graphical user interface (GUI) to be displayed byan electronic display that includes a name of the physical entity, awebsite associated with the physical entity, and one or more inputsconfigured to receive a social networking command. The methodadditionally includes receiving a social networking command for thephysical entity or the website. The method also includes providing anindication of the social networking command to an electronic device.

A further implementation is a social networking system. The systemincludes a processing circuit operative to receive a digital image of aphysical entity. The processing circuit is also operative to identifythe physical entity in the digital image and to provide entity data forthe identified physical entity. The entity data is configured to cause aGUI to be displayed by an electronic display that includes a name of thephysical entity, a website associated with the physical entity, and oneor more inputs configured to receive a social networking command. Theprocessing circuit is further operative to receive a social networkingcommand for the physical entity or the website. The processing circuitis additionally operative to provide an indication of the socialnetworking command to an electronic device.

These implementations are mentioned not to limit or define the scope ofthis disclosure, but to provide examples of implementations to aid inunderstanding thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features,aspects, and advantages of the disclosure will become apparent from thedescription, the drawings, and the claims, in which:

FIG. 1 is an illustration of a user capturing a digital image of aphysical entity;

FIG. 2 is an illustration of a computer system in accordance with adescribed implementation;

FIG. 3 is an example display of a GUI; and

FIG. 4 is an example process for using an image in a social networkingsystem.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

According to some aspects of the present disclosure, a digital image maybe used as part of an online social networking system to allow users toexpress their opinions about entities that exist in the physical world.For example, a user may take a picture of a landmark using a mobiledevice to rate the landmark itself. Image recognition and/or locationdetection may be used in various implementations to identify thephysical entity in the image. A singular entity profile may be storedfor the physical entity, e.g., on a server of the social networkingsystem, allowing multiple users to express their opinions about the sameentity. Information about the identified physical entity may also beprovided using the entity profile. For example, historical informationabout the physical entity may be displayed (e.g., the identity of theentity, when the entity came into being, etc.). In some cases, theopinions of other users about the physical entity may be provided (e.g.,by rating the entity, by commenting about the entity, and the like). Forexample, the opinions of a user's social connections, i.e., other usersin the user's social network, may be displayed to the user.

In some implementations, a user may perform a social networking action(e.g., rating, commenting, sharing, etc.) regarding a website associatedwith a physical entity in a digital image. This may be done in additionto, or in lieu of, actions regarding the physical entity itself. Forexample, a user may take a photograph of a particular restaurant. Therestaurant in the image may be automatically recognized by the system.If the restaurant has an associated website (e.g., a website devoted tothat particular restaurant or to a chain of restaurant), the user mayrate the website. This may be done in addition to, or in lieu of, ratingthe particular restaurant.

Referring to FIG. 1, an illustration of a user 102 capturing a digitalimage of a physical entity 106 is shown, according to one example. User102 may utilize a mobile device 108 to capture the image, according toone implementation. Mobile device 108 may be any form of portableelectronic device having an integrated camera. For example, mobiledevice 108 may be a cellular telephone, a laptop computer, a tabletcomputer, a portable video game player, a digital video recorder, or maysimply be a digital camera. The integrated camera may include acharge-coupled device (CCD), an active-pixel sensor (APS), a photodiode,or any other form of light sensor. The captured digital image may bestored within an electronic memory of mobile device 108. In someimplementations, mobile device 108 may be configured to communicate withother electronic devices. For example, mobile device 108 may communicatewirelessly via a network with one or more other electronic devices(e.g., using a radio, cellular, satellite, or other transceiver). Insome implementations, mobile device 108 may communicate with anotherelectronic device via a hardwired connection (e.g., using an Ethernetcable, using a universal serial bus (USB) cable, or the like).

As shown, user 102 may operate mobile device 108 to capture a digitalimage of physical entity 106 (e.g., the Leaning Tower of Pisa). Forexample, user 102 may operate mobile device 108 to capture a digitalimage of person 104 standing in front of the Leaning Tower of Pisa. Thedigital image may be analyzed to automatically identify physical entity106 in the image. Information about physical entity 106 may be providedto user 102, based on the identification. A social networking system atwhich user 102 has a user profile may also allow user 102 to express anopinion about the physical entity 106. For example, user 102 may ratethe Leaning Tower of Pisa up or down or post a comment about the LeaningTower of Pisa via a social networking system. In another example, user102 may share information about the Leaning Tower of Pisa with socialconnections of user 102.

In some implementations, physical entity 106 may have an associatedwebsite. For example, the Leaning Tower of Pisa may have an officialwebsite at the web address: http://www.example.org/Leaning_Tower.html.In such a case, information about a physical entity identified in adigital image may also include information about the associated website.User 102 may be able to perform a social networking action relating tothe associated website in addition to, or in lieu of, one relating tophysical entity 106 itself.

Referring to FIG. 2, an illustration of a computer system 200 is shown.System 200 may include a server 204 in communication with otherelectronic devices via a network 202. In some implementations, mobiledevice 108 may communicate with server 204 via network 202. Server 204may be part of a social networking system that allows a user to expresstheir opinion about various topics with other users socially connectedto the user (e.g., friends, acquaintances, co-workers, etc.). In furtherimplementations, a client 220 may be in communication with server 204via network 202 and utilized by a user to access the social networkingsystem.

Network 202 may be any form of computer network that relays informationbetween server 204, client 220, and/or mobile device 108. For example,network 202 may include the Internet and/or other types of datanetworks, such as a local area network (LAN), a wide area network (WAN),a cellular network, satellite network, or other types of data networks.Network 202 may also include any number of computing devices (e.g.,computer, servers, routers, network switches, etc.) that are configuredto receive and/or transmit data within network 202. Network 202 mayfurther include any number of hardwired and/or wireless connections. Forexample, client 220 may communicate wirelessly (e.g., via WiFi,cellular, radio, etc.) with a transceiver that is hardwired to othercomputing devices in network 202 (e.g., via a fiber optic cable, a CAT5cable, etc.).

Mobile device 108 may be any form of portable electronic device thatincludes a processor 210 and a memory 212, i.e., a processing circuit.Memory 212 may store machine instructions that, when executed byprocessor 210, cause processor 210 to perform one or more of theoperations described herein. Memory 212 may also store one or moredigital images by the integrated camera of mobile device 108. Processor210 may include a microprocessor, ASIC, FPGA, etc., or combinationsthereof. Memory 212 may include, but is not limited to, electronic,optical, magnetic, or any other storage or transmission device capableof providing processor 210 with program instructions. Memory 212 mayinclude a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM,RAM, EEPROM, EPROM, flash memory, optical media, or any other suitablememory from which processor 210 can read instructions. The instructionsmay include code from any suitable computer programming language suchas, but not limited to, C, C++, C#, Java, JavaScript, Perl, HTML, XML,Python and Visual Basic. According to various implementations, mobiledevice 108 may include a transceiver 206 configured to communicatewirelessly with network 202 via transceiver station 208. For example,transceiver station 208 may be a cellular tower, a mobile hotspot (e.g.,a Wi-Fi access point), a satellite transceiver, or similar mechanismthat provides wireless access to network 202. According to variousimplementations, mobile device 108 may be configured to communicatedirectly with client 220, in addition to, or in lieu of communicatingwith transceiver station 208. For example, mobile device 108 may behardwired to client 220 via a cable (e.g., a USB cable, a parallel portcable, etc.). In another example, mobile device 108 may communicatewirelessly with client 220 via transceiver 206 using a short-rangewireless protocol (e.g., ad-hoc Wi-Fi, Bluetooth, etc.).

Mobile device 108 may include one or more user interface devices,according to various implementations. A user interface device may be anyelectronic device that conveys data to a user by generating sensoryinformation (e.g., a visualization on a display, one or more sounds,etc.) and/or converts received sensory information from a user intoelectronic signals (e.g., a keyboard, a mouse, a pointing device, atouch screen display, a microphone, etc.). The one or more userinterface devices may be internal to the housing of mobile device 108(e.g., a built-in display, microphone, etc.) or external to the housingof mobile device 108 (e.g., a monitor connected to mobile device 108, aspeaker connected to mobile device 108, etc.), according to variousimplementations. For example, mobile device 108 may include anelectronic display 214, which may display a digital image captured bymobile device 108 and/or a GUI for the social networking system ofserver 204.

Client 220 may include of any number of different types of userelectronic devices configured to communicate via network 202 (e.g., alaptop computer, a desktop computer, a tablet computer, a smartphone, adigital video recorder, a set-top box for a television, a video gameconsole, combinations thereof, etc.). Similar to mobile device 108,client 220 may have a processing circuit that includes a processor 222and a memory 224 that stores program instructions executable byprocessor 222. In some implementations, client 220 may include anelectronic display 226, internally or externally, to provide a user witha user interface for the social networking system of server 204 (e.g.,an application stored in memory 224, a website provided by server 204,etc.)

Server 204 may include one or more electronic devices configured tocommunicate with other electronic devices via network 202 and to providethe electronic devices with access to a social networking system. Asocial networking system allows a user to create a user profile andassociate their user profile with that of other users. In some cases,associations between user profiles may be categorized by type. Forexample, a user may categorize other users by their relationship to theuser (e.g., friends, relatives, co-workers, etc.) and/or by commoninterests (e.g., a common love of hockey, a common interest inshipbuilding, etc.). Server 204 may provide a user interface for thesocial networking system to other electronic devices via network 202.For example, server 204 may provide a website to client 220 and/or tomobile device 108. In another example, client 220 and/or mobile device108 may run a local application for the social network that receivesdata from server 204.

Similar to mobile device 108, server 204 may have a processing circuitthat includes a processor 216 and a memory 218. Memory 218 may storeinstructions for execution by processor 216 to perform the functionsassociated with a social networking system. According to variousimplementations, server 204 may be a set of networked devices (e.g., adata center, a plurality of data centers, a cloud computing environment,etc.). In such a case, the processing circuit of server 204 may includethe combined set of processors and memories of the different devices.For example, server 204 may include a webserver that provides websitesto client 220 and/or to mobile device 108 and a back-end database thatstored user profiles and other data associated with the socialnetworking system.

According to various implementations, mobile device 108 and/or client220 may provide a digital image to server 204 via network 202, toidentify a physical entity in the image (e.g., physical entity 106). Forexample, mobile device 108 may capture an image of the physical entityand server 204 may utilize image recognition to identify the physicalentity in the image. In some implementations, other data may also besent to server 204 to identify the physical entity. For example, alatitude, a longitude, and/or a location radius value corresponding tothe location at which the image was captured may be sent to server 204and used to identify the physical entity in the image. For example, thegeographic location at which an image of the Leaning Tower of Pisa wascaptured may be matched by server 204 to the geographic location of theLeaning Tower of Pisa. Such location matching may be used in combinationwith image recognition, or in lieu thereof, to determine that the imageis of the Leaning Tower of Pisa. According to some implementations, theimage recognition and/or geographic location matching may be performedby mobile device 108 and/or client 220. In such a case, mobile device108 or client 220 may provide an indication of the identified physicalentity to server 204, to retrieve information about the identifiedentity and to perform social networking functions associated with thephysical entity.

In some implementations, server 204 may store a unique identifier andadditional information about a physical entity in memory 218. Forexample, server 204 may store information such as the background of thephysical entity (e.g., how the entity was formed or constructed, whenthe entity was formed or constructed, etc.). In one implementation,server 204 may store the location of a website associated with thephysical entity. For example, server 204 may store an official URL forthe Leaning Tower of Pisa.

Server 204 may provide information about a physical entity to mobiledevice 108 and/or to client 220. After the physical entity in a digitalimage has been identified (e.g., by mobile device 108, client 220,server 204, etc.), information about the entity may be retrieved byserver 204 from memory 218 or another location. For example, server 204may combine information about the identified entity stored in memory 218(e.g., an entity profile) with other information stored on variouswebsites. The information about the physical entity may be provided byserver 204 to mobile device 108 and/or client 220. In some cases, server204 may automatically return information about a physical entity, inresponse to receiving a digital image of the entity from mobile device108 and/or client 220 and without requiring any further user input. Inother cases, server 204 may return information about the entity, inresponse to receiving a request for information from mobile device 108or client 220 (e.g., if the physical entity is identified by mobiledevice 108 or client 220).

In one implementation, mobile device 108 and/or client 220 may beutilized to create a new identifier for a physical entity in server 204.A user of mobile device 108 or client 220 may be prompted to enterinformation about a physical entity that cannot be identified in adigital image. For example, server 204 may store an entity profile forthe Leaning Tower of Pisa, but may not have an entity profile for thetown statue in Dunning, Nebr. In such a case, a user of mobile device108 or client 220 may be prompted to enter information about the statue(e.g., the name of the statue, the background of the statue, a websitefor the statue, etc.). The entered information, the digital image,and/or the location information may be stored by server 204 to identifythe statue in the future.

According to various implementations, server 204 may be configured toperform certain social networking functions associated with anidentified physical entity. For example, a user having a socialnetworking profile on server 204 may perform activities such as ratingthe physical entity, commenting on the physical entity, or recommendingthe physical entity to their social connections (e.g., other userprofiles connected to the user's profile via the social networkingsystem). In further implementations, the user may perform socialnetworking activities for a website associated with the identifiedphysical entity. In some cases, server 204 may also provide anindication of social networking activity by others to a user's device.For example, server 204 may provide an indication to mobile device 108or client 220 of the user's social connections rating the physicalentity, commenting on the physical entity, or sharing the physicalentity with the user.

Referring now to FIG. 3, an example display of a GUI 300 is shown,according to a described implementation. GUI 300 may be provided to anelectronic display of a user device used to access a social networkingsystem (e.g., mobile device 108, client 220, etc.). For example, GUI 300may be provided to display 214 of mobile device 108 or to display 226 ofclient 220. In one implementation, GUI 300 may be provided to display214 in response to the physical entity in digital image 315 beingidentified (e.g., by mobile device 108, by client 220, by server 204,etc.). A physical entity in a digital image may be identifiedautomatically when the image is captured or may be identified inresponse to receiving a command from a user interface to do so. Forexample, mobile device 108 may automatically send digital image 315 toserver 204 for identification, when digital image 315 is captured. Inanother example, the user of mobile device 108 may select to have thephysical entity in digital image 315 identified. Digital image 315 mayor may not be provided as part of GUI 300, according to variousimplementations.

GUI 300 may include an identifier 302 for the identified physical entityin digital image 315. For example, the Leaning Tower of Pisa may beidentified in digital image 315 by mobile device 108, by client 220, orby server 204. Identifier 302 may include the name of the physicalentity and/or other identification information for the physical entity(e.g., location coordinates, nicknames for the physical entity, etc.).For example, the name “Leaning Tower of Pisa” may be displayed as partof GUI 300.

GUI 300 may include an input field 304 configured to receive anidentifier for the physical entity in image 315. According to variousimplementations, input field 304 may or may not be provided as part ofGUI 300, based on a confidence score for the identification processbeing below a predetermined score. Such a confidence score may indicatethe likelihood that the identified physical entity is actually thephysical entity in image 315. One factor that may be used to calculate aconfidence score is how closely the location information associated withimage 315 is to that of the identified physical entity. For example, animage taken at the base of the Leaning Tower of Pisa may receive ahigher confidence score that the entity in the image is, in fact, theLeaning Tower of Pisa. Another factor that may be used to calculate theconfidence score is how well the features of the image match those ofthe identified entity. For example, the entity in the image may bepartially obscured by people or other objects, potentially decreasingthe accuracy of image recognition. In various implementations, inputfield 304 may be presented as part of GUI 300 if the confidence score isbelow a confidence threshold. For example, if the confidence scorecorresponds to a 95% likelihood that the identified entity is the entityin the digital image, input field 304 may be hidden. However, if theconfidence score is only 55%, input field 304 may be displayed on GUI300.

Input field 304 may be configured to add a new profile for a physicalentity to the social networking system. In some cases, a physical entitymay not be identified in a digital image because information about theentity is not known by the system. In one implementation, informationabout an unknown physical entity entered into input field 304 may beused to create a new entry for the entity in the memory of the devicethat identifies physical entities. In some implementations, the enteredinformation about the new physical entity may be available to otheruser. For example, a first user may enter information about a physicalentity shown in a digital image. When another user also takes a pictureof the entity, the entity may be automatically identified based on theinformation entered by the first user. In other implementations,information about a new physical entity may be available after a certainnumber of users have identified the same entity. For example, a physicalentity may be automatically identified by the system only after tenusers have manually identified the same entity. Such a feature mayprevent false entries from being available to other users. In furtherimplementations, a user profile may have an associated trustworthinessscore that may be used to determine whether information about a newlyidentified physical entity is available to other users. For example, auser that writes for a travel magazine may have a high trustworthinessscore and their entries may be immediately available to other users.However, an entry from a teenager with a low trustworthiness score maynot be available unless verified by others.

GUI 300 may include information 308 about an identified physical entity.Information 308 may include historical information about the physicalentity, trivia about the physical entity, statistics about the physicalentity, and other information about the identified physical entity. Forexample, if the physical entity in digital image 315 is the LeaningTower of Pisa, information 308 may include information as to how it wasconstructed, its height, and why it was constructed. In someimplementations, information 308 may be entered by a user (e.g., theuser that first identified the Leaning Tower of Pisa in the system, asubsequent user, etc.). In other implementations, information 308 may beaggregated via a web search for information about the physical entity.For example, the device that identifies the Leaning Tower of Pisa indigital image 315 may perform an Internet search to retrieve additionalinformation about the physical entity. In some implementations,information 308 may include one or more hyperlinks to websites thatcontain information about the physical entity (e.g., websites notofficially associated with the physical entity).

GUI 300 may include a rating input 310 configured to receive a ratingfor the physical entity from a user interface device. Rating input 310may be any form of input field to rate an physical entity identified ina digital image. In some implementations, rating input 310 may beconfigured to receive a positive rating for the physical entity. Forexample, rating input 310 may allow a user to simply rate up thephysical entity (e.g., rating input 310 is a single button to positivelyrate the physical entity). According to various implementations, ratinginput 310 may be configured to receive a negative rating for thephysical entity, in addition to, or in lieu of, being configured toreceive a positive rating. In some implementations, a rating may beclassified by degree. For example, a physical entity may be rated by auser on a sliding scale to indicate how much the user likes orrecommends the physical entity. In such cases, rating input 310 may be atext field (e.g., allowing users to enter a score ranging from A-F, from1-10, etc.), a sliding bar, one or more checkboxes, a selectable radialfield, or any other mechanism configured to allow a user to specify arelative strength of a rating for the physical entity.

GUI 300 may include a field 314 that provides an indication of whichother users also rated the physical entity. For example, users “JohnDoe” and “Jane Doe” may have positively rated the Leaning Tower of Pisa.In such a case, an indication of one or more of the users that rated thephysical entity may be provided as part of GUI 300 in field 314. In someimplementations, the users identified in field 314 may be connected withthe social networking profile of the user operating the devicedisplaying GUI 300. For example, the user of mobile device 108 may belogged into their user profile on the social networking system providedby server 204. In some implementations, which users are presented infield 314 may be based in part on their relationship to the user ofmobile device 108 via the social networking system. For example, usersclassified as friends of the user may take priority over that of theuser's coworkers. In various implementations, field 314 may display thetotal number of ratings (e.g., positive ratings, negative ratings, etc.)for the physical entity. For example, field 314 may display the totalnumber of positive ratings by a user's social connections or by allusers of the social networking system. In one implementation, a totalscore may be provided, if users are able to rate the physical entityusing a range of values. For example, the physical entity may receive anaverage rating of 9.5 out of a possible 10.

GUI 300 may include a sharing input 320, in some implementations.Sharing input 320 may be configured to allow the user of the device toshare information about the identified physical entity to one or moresocial connections via the social networking system. In someimplementations, sharing input 320 may be configured to allow a user tospecifically recommend the identified physical entity to certain users(e.g., a particular user, a group of users, all social connections ofthe user, etc.). For example, sharing input 320 may be used to recommendthe Leaning Tower of Pisa to the user's sister or to a social groupdevoted to tourism of which the user belongs. Sharing a physical entitywith a social connection may cause a notification to be received by asocial connection or may cause information about the physical entity toappear in the user's newsfeed. According to various implementations, thefunctions associated with sharing input 320 may be integrated with thatof rating input 310 or vice versa. For example, a user positively ratingthe Leaning Tower of Pisa via rating input 310 may cause a socialconnection of the user to receive information about the Leaning Tower ofPisa.

GUI 300 may include a comment field 322 configured to receive a commentabout the identified physical entity. For example, the user of mobiledevice 108 may provide a comment about the Leaning Tower of Pisa toother users of the social networking system via comment field 322. Acomment received via comment field 322 may be provided to other users ofthe social networking system. For example, a comment entered via commentfield 322 may appear in the user's newsfeed, on a displayed fieldassociated with the physical entity (e.g., in field 314 of GUI 300), ona profile page for the physical entity, or sent as a notification to oneor more social connections of the user.

According to various implementations, GUI 300 may be configured toprovide information about a website associated with an identifiedphysical entity. For example, the Leaning Tower of Pisa may have anofficial website at the URL: http://www.example.org/Leaning_Tower.html.In such a case, GUI 300 may include a website identifier field 316. Oncea physical entity has been identified (e.g., by using image recognitionon digital image 315 and/or using location information associated withdigital image 315), a URL or other identifier for a website associatedwith the entity may be retrieved and displayed as part of GUI 300. Insome implementations, website identifier field 316 may include ahyperlink to the website (e.g., website identifier field 316 may beselected to navigate to the identified website).

In implementations in which an identified physical entity has anassociated website, GUI 300 may be configured to provide socialnetworking functionality for the website in a manner similar to that ofthe physical entity itself For example, GUI 300 may include a ratinginput 312 to rate the website associated with the identified physicalentity. GUI 300 may also include field 318 configured to displayratings, comments, or other activity related to the website by socialconnections of the user. Similarly, GUI 300 may include a sharing input324 for sharing the website and/or a comment field 326 for entering acomment about the website.

In further implementations, a physical entity may be part of a group ofrelated physical entities. For example, the restaurant chain, “ThankGoodness for Mondays!” may have locations in Las Vegas, Seattle, andChicago. In such a case, GUI 300 may allow a user to enter socialnetworking commands (e.g., to share, rate, comment on, etc.) regardingthe group of entities and/or an individual physical entity. For example,the user of mobile device 108 may take a picture of the restaurantlocation in Las Vegas. In response, GUI 300 may provide informationabout that particular restaurant location and/or may provide informationabout the entire restaurant chain. Similarly, GUI 300 may be configuredto receive commands to perform social networking functions relating tothe Las Vegas location or to the restaurant chain itself For example,the user may positively rate the Las Vegas location because of itsarchitecture or may comment on the entire restaurant chain (e.g., thatthey enjoy eating at Thank Goodness for Mondays! restaurants, etc.).

Input received via GUI 300 may be associated with a user profile of asocial networking system. For example, the user operating mobile device108 may log into their user profile in a social networking system, toretrieve and share information about the Leaning Tower of Pisa withothers. Input received via rating inputs 310, 312, sharing inputs 320,324, and/or comment fields 322, 326 may be provided to a server of thesocial networking system. For example, a user of mobile device 108 mayutilize comment field 322 to provide a comment about the Leaning Towerof Pisa with others via the server of the social networking system.

Referring to FIG. 4, an example process 400 for using an image in asocial networking system is shown. Process 400 includes receiving animage of a physical entity (block 402). The image may be captured, forexample, by a portable electronic device having an integrated camera. Insome implementations, the image may be received at the portableelectronic device capturing the image. For example, the digital imagemay be received in response to operation of a CCD, APS, or other form oflight sensing mechanism integrated into the portable electronic device.In other implementations, the digital image may be received at a serveror other client device. For example, the digital image may be providedto a home computer or other consumer electronic device by the portableelectronic device that captured the image. In another example, thedigital image may be received by a server of a social networking systemdirectly from the portable electronic device or indirectly (e.g., thedigital image is first uploaded to a client device and then sent to theserver).

Process 400 includes identifying a physical entity in the image (block404). In some implementations, the device that receives the digitalimage may be configured to retrieve information about a physical entityfrom a database. For example, a server of a social networking system mayinclude a database of physical entity identifiers (e.g., entity names,unique numbers, etc.) and associated characteristics of the physicalentity (e.g., location information for the physical entity, relativesize or shape information for the physical entity, etc.), as part of aphysical entity profile in the social networking system. In someimplementations, image recognition may be performed on the receivedimage to identify the physical entity in the image. For example, thecharacteristics of the physical entity in the image (e.g., shape,proportionate size, coloration, etc.) may be matched to those in thedatabase, to identify the physical entity. In some implementations,location information associated with the image may be used as part ofthe identification. For example, the latitude, longitude, and/or radiusof the location at which the image was captured may be used to identifythe physical entity. The system may also use the direction that themobile device was facing (e.g., from an electronic compass on thedevice) to assist in identifying the physical entity. In someimplementations, a confidence score may be calculated to denote thelikelihood that the identified physical entity is actually the physicalentity in the image. The confidence score may be calculated, forexample, based on how closely the characteristics in the database matchthose of the image.

Identification of the physical entity may be performed by a server of asocial networking system, a client device, or by the portable electronicdevice that captured the image, according to various implementations.For example, the digital image may be sent to a server from a mobiletelephone, to perform image recognition on the image. Alternatively, themobile telephone itself may be configured to identify a physical entityin a digital image. For example, the mobile telephone may include adatabase of physical entities or may query a database of physicalentities via a network. In another example, a digital image may beuploaded to a client device configured to identify physical entities.

Process 400 includes providing entity data to a GUI (block 406). Entitydata is to be understood as any information regarding a physical entity(e.g., a name of the entity, background information about the entity,the location of the entity, trivia about the entity, an official websiteassociated with the entity, websites that contain further informationabout the entity, etc.). After the physical entity has been identified,entity data may be retrieved and displayed. In some implementations,entity data may be stored with the characteristics used to identify thephysical entity. In other implementations, the identificationinformation may be stored separately from that of other informationabout the entity. For example, a mobile phone may identify a physicalentity using a local database and provide an indication of theidentified entity to a remote server, to retrieve entity data for thephysical entity.

According to various implementations, the GUI may be a website, astandalone application, or a combination thereof. For example, the GUImay be a website provided by a server of the social networking system.The server may retrieve the entity data and provide a website containingthe data to a user device. In another example, the GUI may be part of aprogram running on a user device. For example, the GUI may be part of asoftware application used to capture the digital image. In such a case,the information about the physical entity may be provided by a localdatabase or by a remote server to the GUI.

Process 400 includes receiving a social networking command regarding thephysical entity and/or a website associated with the physical entity(block 408). A social networking command refers to any command toperform an action associated with a user profile of a social networkingsystem. For example, a social networking command may correspond to acommand from a user to positively rate the physical entity or website,store a comment on the physical entity or website, or share informationabout the physical entity or website with others via the socialnetworking system. The social networking command may be received, forexample, from a user interface device (e.g., a keypad, a pointingdevice, a touch screen display, etc.) at a client device or a portableelectronic device. In another example, the social networking command maybe received at a server of the social networking site from anotherelectronic device (e.g., a client device, a portable electronic device,etc.).

Process 400 includes providing an indication of the social networkingcommand to another device (block 410). The indication may be provided toa server of the social networking system or to an electronic deviceoperated by a user, according to various implementations. In cases inwhich the social networking command is received at a server of a socialnetworking system, the server may provide an indication of theassociated action to another user electronic device. For example, a usermay input a comment on the physical entity using a first device. Thecomment may be provided to a server of the social networking system. Theserver may provide an indication of the comment to a second deviceoperated by a second user (e.g., a social connection of the first user).For example, the comment itself may be provided to the second device ora notification that the first user entered a comment may be provided. Incases in which the social networking command is received at a userelectronic device, the user electronic device may provide an indicationof the action to a server of the social networking system.

In one example of process 400, a user may capture a digital image of aphysical entity using their mobile telephone. The image may betransmitted from the mobile telephone to a server of a social networkingsystem. The server, in turn, may use the digital image to identify thephysical entity in the image and to retrieve information about theidentified entity. The server may then transmit the retrievedinformation about the entity to the mobile telephone, which displays theinformation as part of a GUI. The user of the mobile telephone may thenutilize the GUI to positively rate the physical entity and/or a websiteassociated with the physical entity. The server of the social networkingsystem may receive the positive rating from the mobile telephone andprovide an indication of the rating to one or more other devices (e.g.,user devices operated by other users of the social networking system).

Implementations of the subject matter and the operations described inthis specification can be implemented in digital electronic circuitry,or in computer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations of the subjectmatter described in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on one or more computer storage medium forexecution by, or to control the operation of, data processing apparatus.Alternatively or in addition, the program instructions can be encoded onan artificially-generated propagated signal, e.g., a machine-generatedelectrical, optical, or electromagnetic signal, that is generated toencode information for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium canbe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate components or media (e.g., multiple CDs, disks, or otherstorage devices). Accordingly, the computer storage medium may betangible and non-transitory.

The operations described in this specification can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

The term “client or “server” include all kinds of apparatus, devices,and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing. The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application-specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an opesocial networking system, a cross-platformruntime environment, a virtual machine, or a combination of one or moreof them. The apparatus and execution environment can realize variousdifferent computing model infrastructures, such as web services,distributed computing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto-optical disks, or optical disks.However, a computer need not have such devices. Moreover, a computer canbe embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a USB flash drive), to name just a few. Devices suitable forstoring computer program instructions and data include all forms ofnon-volatile memory, media and memory devices, including by way ofexample semiconductor memory devices, e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,special purpose logic circuitry.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube), LCD (liquidcrystal display), OLED (organic light emitting diode), TFT (thin-filmtransistor), plasma, other flexible configuration, or any other monitorfor displaying information to the user and a keyboard, a pointingdevice, e.g., a mouse, trackball, etc., or a touch screen, touch pad,etc., by which the user can provide input to the computer. Other kindsof devices can be used to provide for interaction with a user as well;for example, feedback provided to the user can be any form of sensoryfeedback, e.g., visual feedback, auditory feedback, or tactile feedback;and input from the user can be received in any form, including acoustic,speech, or tactile input. In addition, a computer can interact with auser by sending documents to and receiving documents from a device thatis used by the user; for example, by sending websites to a web browseron a user's client device in response to requests received from the webbrowser.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front-endcomponent, e.g., a client computer having a GUI or a Web browser throughwhich a user can interact with an implementation of the subject matterdescribed in this specification, or any combination of one or more suchback-end, middleware, or front-end components. The components of thesystem can be interconnected by any form or medium of digital datacommunication, e.g., a communication network. Examples of communicationnetworks include a local area network (“LAN”) and a wide area network(“WAN”), an inter-network (e.g., the Internet), and peer-to-peernetworks (e.g., ad hoc peer-to-peer networks).

The features disclosed herein may be implemented on a smart televisionmodule (or connected television module, hybrid television module, etc.),which may include a processing circuit configured to integrate internetconnectivity with more traditional television programming sources (e.g.,received via cable, satellite, over-the-air, or other signals). Thesmart television module may be physically incorporated into a televisionset or may include a separate device such as a set-top box, Blu-ray orother digital media player, game console, hotel television system, andother companion device. A smart television module may be configured toallow viewers to search and find videos, movies, photos and othercontent on the web, on a local cable TV channel, on a satellite TVchannel, or stored on a local hard drive. A set-top box (STB) or set-topunit (STU) may include an information appliance device that may containa tuner and connect to a television set and an external source ofsignal, turning the signal into content which is then displayed on thetelevision screen or other display device. A smart television module maybe configured to provide a home screen or top level screen includingicons for a plurality of different applications, such as a web browserand a plurality of streaming media services, a connected cable orsatellite media source, other web “channels”, etc. The smart televisionmodule may further be configured to provide an electronic programmingguide to the user. A companion application to the smart televisionmodule may be operable on a mobile computing device to provideadditional information about available programs to a user, to allow theuser to control the smart television module, etc. In alternateembodiments, the features may be implemented on a laptop computer orother personal computer, a smartphone, other mobile phone, handheldcomputer, a tablet PC, or other computing device.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking orparallel processing may be utilized.

1. A computerized method for using a digital image in a socialnetworking system comprising: receiving, at a processing circuit, arequest to log into a user profile of the social networking system;receiving, at the processing circuit, a digital image of a physicalentity; associating the digital image with the user profile; performing,by the processing circuit, image recognition on the digital image toidentify the physical entity; retrieving, from a memory, the name of thephysical entity and an indication of a website associated with thephysical entity; providing entity data over a network, wherein theentity data is configured to cause a graphical user interface to bedisplayed by an electronic display, the graphical user interfacecomprising the name of the physical entity, the indication of a websiteassociated with the physical entity, and one or more inputs configuredto receive a rating for the website associated with the physical entity;receiving, at the processing circuit, the rating for the websiteassociated with the physical entity, the rating being associated withthe user profile; and providing an indication of the rating for thewebsite to an electronic device based on whether the electronic deviceis logged into a user account socially connected to the user accountassociated with the rating. 2-20. (canceled)